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1.  Introduction 


1.1.  Background 

The  former  Erie  Army  Depot,  Ottawa  County,  Ohio,  is  located  along  the  western  shore  of  Lake 
Erie.  This  site  and  the  associated  impact  areas  are  classified  by  the  United  States  Government  as 
Formerly  Used  Defense  Sites  (FUDS)  under  the  Defense  Environmental  Restoration  Program 
(DERP).  This  property  was  formerly  used  for  artillery  testing,  resulting  in  impact  areas  on  land 
and  in  Lake  Erie.  Munitions  and  explosives  of  concern  (MEC),  including  potentially  live  or 
unexploded  ordnance  (UXO)  have  been  found  on  the  lake  bottom,  in  the  Federal  navigation 
channel  at  the  Toussaint  River,  in  the  marshland  adjacent  to  the  firing  ranges,  and  along  beaches 
fronting  the  former  Depot.  The  impact  areas  were  located  in,  near,  or  offshore  of  the  FUDS 
beaches  adjacent  to  Lake  Erie.  Ordnance  found  on  or  near  the  FUDS  shore  of  Lake  Erie  appears 
to  be  mobile  and  may  have  originated  from  offshore  or  nearshore  impact  areas.  In  FY06,  the 
Environmental  Security  Technology  Certification  Program  (ESTCP)  was  directed  by  Congress  to 
conduct  work  to  characterize  UXO  contamination  impacting  the  Toussaint  River  area,  and  this 
survey  is  part  of  ESTCP’s  Wide  Area  Assessment  Pilot  Program  (WAA-PP). 

This  demonstration  utilized  Helicopter  Multi-Sensor  Towed  Array  Detection  System  (MTADS) 
Magnetometry  (HeliMag)  technology,  a  wide  area  assessment  (WAA)  technology,  to  assist  in  the 
characterization  of  the  shore  and  shallow  areas  in  and  around  the  Toussaint  River  relative  to 
munitions  contamination  from  historical  activities  at  the  Erie  Army  Depot  and  Camp  Perry. 
HeliMag  provides  efficient  low  altitude  digital  geophysical  mapping  (DGM)  capabilities  for 
metal  detection  and  feature  discrimination  at  a  resolution  approaching  that  of  ground  survey 
methods,  limited  primarily  by  terrain,  vegetation  and  structural  inhibitions  to  safe  low-altitude 
flight.  This  demonstration  was  conducted  under  ESTCP  MM-0535. 

1.2.  Objectives  of  the  Demonstration 

The  purpose  of  this  demonstration  was  to  survey  marsh,  shallow  water  and  river  areas  at  the  site 
as  described  by  this  demonstration  plan.  Specific  objectives  of  this  demonstration  included: 

•  Characterize  the  marsh,  shallow  water  and  river  areas  in  and  around  the  Toussaint  River 
that  are  contaminated  with  munitions  from  historical  activities  at  the  Erie  Army  Depot 
and  Camp  Perry,  including: 

o  Identification  of  areas  of  concentrated  munitions; 

o  Bound  the  munitions-contaminated  areas  in  Lake  Erie  and  on  the  adjacent  beaches; 

o  Estimate  density  and  distribution  of  munitions  types  and  sizes; 

o  Characterize  site  conditions  to  support  future  investigation,  prioritization, 
remediation,  and  cost  estimation  tasks. 
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A  determination  of  success  for  this  demonstration  was  based  on  the  performance  of  the  system, 
as  described  in  Section  4. 


1.3.  Regulatory  Drivers 

The  Toussaint  River  is  situated  immediately  north  of  the  former  Erie  Army  Depot.  This  river 
includes  several  small  craft  and  commercial  fishing  marinas.  The  U.S.  Army  Corps  of  Engineers 
(USACE)  -  Buffalo  District  is  responsible  for  maintaining  a  navigable  waterway.  The  area  is 
periodically  dredged,  either  by  the  USACE  or  local  private  interests.  These  dredging  activities 
are  often  affected  by  munitions  encounters.  Figure  1  shows  the  approximate  survey  area 
boundaries  and  mouth  of  the  Toussaint  River.  This  site  and  the  associated  impact  areas  are 
classified  by  the  United  States  Government  as  a  FUDS  under  the  DERP. 


Toussaint  River 
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Figure  1.  Former  Erie  Army  Depot  and  vicinity  with  HeliMag  survey  area  outlined  in  red. 
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1.4.  Stakeholder/End-User  Issues 

ESTCP  is  managing  the  stakeholder  issues  as  part  of  its  WAA-PP.  ESTCP  plans  to  use  a  process 
that  will  ensure  that  the  information  generated  by  the  helicopter,  water  and  validation  surveys  is 
useful  to  a  broad  stakeholder  community  (e.g.,  technical  project  managers  and  Federal,  State,  and 
local  governments,  as  well  as  other  stakeholders). 
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2.  Technology  Description 

2.1.  Technology  Development  and  Application 

The  Sky  Research,  Inc.  helicopter  technology  is  based  on  the  Naval  Research  Laboratory  (NRL) 
MTADS  technology,  transferred  to  Sky  Research  for  commercialization  via  a  Cooperative 
Research  and  Development  Agreement  (CRADA).  Prior  to  the  transfer,  this  technology  was  fully 
evaluated  by  the  Department  of  Defense  (DoD)  by  ESTCP  (Nelson  et  ah,  2005;  Tuley  and 
Dieguez,  2005). 

The  HeliMag  system  includes  a  helicopter-borne  array  of  magnetometers  and  software  designed 
specifically  to  process  data  collected  with  this  system  and  perform  physics-based  analyses  on 
identified  targets.  In  addition,  Sky  Research  recently  completed  updates  to  the  NRL  MTADS 
technology  to  improve  performance  and  reliability  of  the  technology. 

2.1.1.  Helicopter  Platform 

Sky  Research  used  a  Hughes  MD530F  helicopter  (Figure  2)  for  data  collection  at  the  Toussaint 
River  site,  which  has  greater  power  and  lift  than  other  helicopters  of  its  size.  Table  1  presents  the 
airborne  MTADS  sensor  system  technologies  deployed  on  the  helicopter  platform,  including: 

•  Array  of  seven  Geometries  822  Cesium  (Cs)  vapor  magnetometers 

•  2  Trimble  MS750  global  position  system  (GPS)  receivers 

•  1  Optech  Laser  altimeter 

•  4  Acoustic  altimeters 

•  Data  acquisition  system  (DAS) 


Figure  2.  Sky  Research,  Inc.  MD  530F  helicopter  and  magnetometer  boom  assembly.  Seven 
magnetometers  are  contained  in  the  composite-material  boom  and  positioned  with  the  GPS 
antenna  and  laser  /  acoustics  altimeters. 
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Table  1.  Sky  Research  HeliMag  Components 


Technology  Component 

Specifications 

Geophysical  Sensors 

7  Geometries  822  Cs  vapor 
magnetometers,  0.001  nanotesla 
(nT)  resolution 

GPS  Equipment 

2  Trimble  MS750  GPS  receivers, 

2-3  centimeter  (cm)  horizontal 
precision 

Altimeters 

1  Optech  laser  altimeter  and  4 
acoustic  altimeters,  1  cm  resolution 

DAS 

Sky  Research  DAS  capable  of  data 
collection  up  to  400  hertz  (Hz), 

10  microsecond  (pSec)  timing 
precision 

Aircraft 

Hughes  MD  53  OF  helicopter 

2.1.2  Sensors  and  Boom 

The  MTADS  magnetic  sensors  are  seven  Geometries  822A  Cs  vapor  full-field  magnetometers 
(the  822 A  is  a  variant  of  the  Geometries  822  sensor).  The  array  of  seven  sensors  is  interfaced  to 
a  DAS  and  the  sensors  are  evenly  spaced  at  1 .5  m  intervals  on  a  9  m  Kevlar  boom  mounted  on 
the  helicopter.  The  new  boom  design  (Figure  3)  updates  the  NRL  boom  design,  with  changes 
including  boom  placement  closer  to  the  helicopter  to  minimize  weight  and  a  quick  assembly 
design  that  decreases  assembly  time  to  two  hours.  Additional  changes  in  boom  design  include 
GPS  sensors  moved  to  7.5  m  separation  or  3.75  m  from  centerline  to  improve  fuselage 
shadowing  and  new  digital  acoustic  sensors,  which  eliminate  the  voltage  devices  in  the  NRL 
system  because  of  their  plastic  housing,  and  which  have  been  moved  to  4.5  m  separation  or 
2.25  m  from  centerline. 


2.1.3  Positioning  Technologies 

Sensor  positioning  is  provided  using  real-time  kinematic  (RTK)  GPS  navigation,  with  real-time 
position  updates  at  20  Hz  and  horizontal  accuracy  of  about  5  cm;  the  Inertial  Measurement  Unit 
(IMU)  is  used  to  correct  for  platform  pitch  and  roll.  At  typical  1-3  m  above  ground  level  (AGL) 
operating  heights,  the  5  cm  RTK  GPS  accuracy  has  been  shown  on  previous  deployments  to 
translate  to  a  horizontal  positioning  error  of  about  <  5  cm  root  mean  square  error  (RMSE). 

The  GPS  satellite  clock  time  is  used  to  time-stamp  both  position  and  sensor  data  information  for 
merging  channel  and  position  data.  An  onboard  navigation  guidance  display  provides  pilot 
guidance,  with  survey  parameters  established  in  a  navigation  computer  that  shares  the  RTK  GPS 
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positioning  data  stream  with  the  DAS.  Pilot  steering  information  (horizontal  and  vertical)  is 
provided  in  graphic  format.  The  GPS  and  altimeter  data  integrity  is  presented  as  a  binary  status 
button  and  the  GPS  fix  quality  is  indicated  via  text  box  as  well  as  by  changing  the  color  of  the 
graphic  position  trace.  This  allows  the  operator  to  respond  to  both  visual  cues  on  the  ground  and 
to  the  survey  guidance  display.  Following  a  survey,  the  operator  can  survey  any  missed  areas 
before  leaving  the  site. 


Figure  3.  HeliMag  sensor  boom  diagram  (2nd  GPS  antenna  not  shown). 
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2.1.4  Data  Acquisition  System 

A  new  DAS  was  developed  by  Sky  Research  for  use  with  the  helicopter  system  in  early  2006 
(Figure  4),  providing  the  following  advantages  over  the  previous  DAS  used  for  WAA-PP 
projects:  smaller  footprint  (3.5”  x  5”  x  6”),  Linux  operating  system  (Realtime  Linux  2.6),  more 
accurate  time  stamping  (10  pS)  and  faster  sampling  rate  (400  Hz  vs.  100  Hz).  The  magnetometer 
data,  logged  at  400  Hz  were  down-sampled  to  100  Hz,  providing  a  nominal  down-the-track 
sample  interval  of  0.15  -  0.20  m  per  sample  at  a  survey  speed  of  15  -  20  meters  per  second  (m/s) 
(30  -  40  knots  [kts]). 


Figure  4.  Sky  Research,  Inc.  DAS. 

2.1.5  Data  Processing 

UXOLab  software  was  used  for  all  HeliMag  data  processing;  this  software  contains  all  the 
functionality  required  to  process  raw  geophysical  data,  detect  anomalous  regions  and  perform 
geophysical  inversions.  During  the  first  data  processing  stage,  the  raw  data  for  a  given  survey 
flight  are  time-aligned  and  transcribed  from  the  various  raw  data  files  into  a  ‘flight’  database. 
Routines  are  run  to  automatically  reject  or  ‘default’  invalid  data.  Data  are  rejected  based  upon 
status  flags  present  in  the  raw  data  records  or,  in  the  case  of  the  magnetometer  data,  a  simple  ‘in 
range’  test  may  be  used.  The  GPS  geographic  position  coordinates  are  transformed  to  WGS84 
Universal  Transverse  Mercator  (UTM)  coordinates.  At  this  point  the  data  are  visually  inspected 
to  ensure  both  integrity  and  quality.  This  pre-processing  stage  is  instrumentation-specific  and  the 
steps  required  to  transcribe  these  data  into  a  time-aligned  database  are  dictated  by  the  structure  of 
the  data  outputs  from  each  device  and  the  manner  in  which  they  are  logged.  All  data  outputs  are 
received  by  the  on-board  DAS.  A  DAS  time  stamp  is  appended  to  each  sample  data  string  and 
the  sample  are  then  stored  in  a  separate  data  file  for  each  device. 
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Data  processing  with  the  use  of  UXOLab  greatly  speeds  up  the  merging  and  data  interpolating 
process  due  to  the  large  database  functionality  and  optimized  merging  algorithms.  Typical 
production  processing  for  300-500  acres  takes  approximately  eight  hours  of  data  processing  to 
produce  a  raw  data  plot  image.  This  image  is  used  to  check  survey  coverage  and  operation  of  the 
system,  and  is  not  typically  used  for  target  picking.  Additional  processing  steps  after  this  raw 
data  step  include  filtering,  geologic  trend  removal,  and  smoothing  if  needed.  Filters  are  used  in 
the  processing  of  magnetic  data  to  mitigate  complicating  effects  in  the  data  and  increase 
capability  to  detect  discrete  buried  targets  and  geologic  trend  removal  is  used  if  geological  trends 
are  present.  Merged  HeliMag  sensor  and  position  data  files  are  then  converted  to  magnetic 
anomaly  maps  using  UXOLab. 

2.1.6  Data  Analysis 

Automatic  target  selection  for  large  scale  surveys  such  as  this  one  has  the  advantage  of  being 
objective  and  repeatable  as  well  as  much  faster  than  manual  selection  if  a  very  large  number  of 
targets  are  to  be  selected.  However,  automatic  target  pickers  are  not  yet  sophisticated  enough  to 
reliably  detect  closely  spaced  targets  or  targets  that  are  at  or  below  the  same  amplitude  as  local 
geologic  signal.  Furthermore  these  automated  routines  are  not  able  to  differentiate  between  our 
targets  of  interest  and  local  geologic  anomalies,  or  response  from  non-UXO  like  anthropologic 
sources  (e.g.  pipelines).  In  practice,  the  decision  to  pick  manually,  or  use  an  auto-picker  then 
add/reject  targets  manually,  is  made  based  upon  the  number  of  targets  to  be  picked  and  the  extent 
of  geologic/anthropologic  clutter. 

For  the  purposes  of  WAA  where  the  main  goal  is  to  delineate  target  density  throughout  the 
survey  site,  the  limitations  of  automatic  target  selection  are  not  as  detrimental  as  they  would  be  if 
we  were  concerned  with  detecting  every  possible  UXO  target.  The  challenge  is  to  calibrate  the 
automatic  target  selection  routine  so  that  the  number  of  valid  targets  of  interest  selected  is 
maximized,  while  minimizing  the  number  of  targets  selected  due  to  geologic  noise  or  other  noise 
sources  (geologic  noise  is  usually  the  predominant  noise  source).  To  achieve  this,  a  subset  of 
manual  target  selection  results  are  compared  with  those  obtained  using  an  automated  target 
selection  routine  over  a  representative  subset  of  the  survey  site. 

The  automatic  selection  process  utilizes  the  Automated  Wavelet  Detection  Algorithm  (Billings 
and  Herrmann,  2003).  Individual  peaks  in  the  magnetic  data  are  followed  across  multiple  scales, 
with  the  decay  in  peak  amplitude  related  to  the  depth  to  the  source.  Nearby  positive  and  negative 
peaks  in  the  image  are  joined  together  if  they  have  comparable  depth  estimates.  In  this  way, 
incorrectly  joining  the  peaks  from  nearby  dipoles  that  are  at  different  depths  can  be  avoided.  In 
the  last  stage  of  the  algorithm,  the  amplitudes  of  the  peaks  and  their  relative  position  are  used  to 
provide  an  initial  estimate  of  the  dipole  parameters.  The  detection  thresholds  are  selected  based 
upon  a  comparison  with  manually  selected  targets  as  well  as  the  objectives  and  limitations 
imposed  by  the  conditions  of  each  specific  site. 
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Filtered  sensor  data  points  were  interpolated  into  magnitude  surface  rasters  using  geographic 
information  system  (GlS)-based  interpolation  scripts.  In  addition,  inversions  were  completed  on 
targets  selected  by  the  ESTCP  Program  Office  using  UXOLab  to  model  the  targets  as  magnetic 
dipoles  controlled  by  six  parameters;  target  X,  Y  location,  depth,  azimuth,  inclination,  and  size. 
The  size  is  measured  as  the  strength  of  the  dipole  source  causing  the  observed  magnetic  anomaly 
and  is  correlated  to  the  specific  targets  of  interest. 

2.2.  Previous  Testing  of  the  Technology 

Previous  testing  of  the  helicopter  magnetometry  technology  in  general  was  supported  by  ESTCP 
(Nelson  et  al.,  2005).  The  primary  development  objective  was  to  provide  a  UXO  site 
characterization  capability  for  extended  areas,  while  retaining  substantial  detection  sensitivity  for 
individual  UXO.  The  system  included  data  collection  hardware  in  the  form  of  a  helicopter-borne 
array  of  magnetometers  and  software  designed  specifically  to  process  data  collected  with  this 
system  and  perform  physics-based  analyses  on  identified  targets. 

2.3.  Factors  Affecting  Cost  and  Performance 

For  all  airborne  surveys,  the  largest  single  factor  affecting  the  survey  costs  is  the  cost  of 
operating  the  survey  aircraft  and  sensors  at  the  site.  These  equipment  costs  are  related  to  capital 
value,  maintenance  overhead  and  direct  operating  costs  of  these  expensive  sensor  and  aircraft 
systems.  Mobilization  to  and  from  the  site  increases  costs  as  distance  increases,  and  flexibility  of 
scheduling  is  critical  in  determining  whether  mobilization  and  deployment  costs  can  be  shared 
across  projects.  In  addition,  helicopter  surveys  are  limited  by  topography  and  vegetation  and 
therefore  can  be  deployed  only  to  sites  with  suitable  conditions. 

Another  significant  cost  factor  is  data  volume  and  the  requirement  for  a  robust  data  processing 
infrastructure  to  manage  large  amounts  of  digital  remote  sensing  data. 

2.4.  Advantages  and  Limitations  of  the  Technology 

As  with  all  characterization  technologies,  site  specific  advantages  and  disadvantages  exist  that 
dictate  the  level  of  success  of  their  application. 

Advantages  of  HeliMag  technologies  include: 

•  The  ability  to  characterize  very  large  areas; 

•  Lower  cost  as  compared  to  ground  based  DGM  methods. 

Limitations  of  HeliMag  technologies  include: 

•  Asa  WAA  tool,  not  intended  to  detect  individual  MEC  items; 

•  Site  physiography,  such  as  terrain,  soils,  and  vegetation,  can  constrain  the  use  of  the 
technology; 
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•  Limited  to  shallow  water  areas  due  to  height  above  target  limitations  of  the 
technology. 
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3.  Demonstration  Design 

3.1.  Performance  Objectives 

Performance  objectives  are  a  critical  component  of  the  demonstration  plan  because  they  provide 
the  basis  for  evaluating  the  performance  and  costs  of  the  technology.  For  the  WAA  projects,  both 
primary  and  secondary  performance  objectives  have  been  established.  Table  2  lists  the 
performance  objectives  for  the  helicopter  MTADS  technology,  along  with  criteria  and  metrics 
for  evaluation,  to  be  documented  as  a  deliverable  of  this  demonstration. 

3.2.  Selecting  Test  Site 

ESTCP  received  Congressional  direction  to  study  the  Toussaint  River  area.  From  historic 
activities  at  Camp  Perry  and  the  former  Erie  Army  Depot,  there  is  a  large  impact  area  in  the 
water.  In  addition,  there  are  other  impact  areas  in  the  swampy  land  adjacent  to  these  facilities  and 
Lake  Erie.  The  primary  interest  of  the  community  is  in  the  UXO  that  affects  dredging  of  the 
navigation  channel  to  the  river.  The  helicopter  technology  was  demonstrated  on  areas  of  the  site 
that  were  conducive  to  low  altitude  magnetometry  in  the  near  shore  area,  the  mouth  of  the 
Toussaint  River,  and  the  shallow  water  areas  of  Lake  Erie.  These  areas  encompassed 
approximately  3,300  acres.  An  underwater  system  was  demonstrated  concurrently  at  the  site  and 
surveyed  transects  in  deeper  water  areas. 

3.3.  Test  Site  History/Characteristics 

Camp  Perry  was  established  in  1907  by  the  state  of  Ohio  for  the  training  of  the  state  National 
Guard.  The  Erie  Army  Depot  was  initially  established  in  1918  on  a  portion  of  the  Camp  Perry 
lands  as  the  Camp  Perry  Proving  Grounds,  and  then  designated  as  the  Erie  Proving  Grounds  in 
support  of  increased  activities  during  the  World  War  II  period.  Between  1918  and  1966,  this  site 
was  used  by  the  U.S.  Army  for  testing  and  proof-firing  artillery  and  as  an  ordnance  storage  and 
issue  center.  During  that  time,  in  1951,  it  was  renamed  the  Erie  Army  Depot  and  assumed  the 
additional  roles  of  anti-aircraft  support  testing  and  the  overhauling  of  surface-to-air  guided 
missiles  during  the  Korean  War.  Ordnance  tested  included  small  caliber  items,  artillery,  mortar 
shells  up  to  240  mm,  and  rockets. 

Activities  to  remove  MEC  by  the  US  ACE  in  the  early  1990s  revealed  that  ordnance  were  being 
transported  by  lake  and  river  waters  to  locations  outside  the  impact  zones.  Dredging  activities  to 
maintain  the  navigable  channel  of  the  Toussaint  River  have  exposed  MEC  (Pope  et  al.,  1996). 

A  more  complete  description  of  the  history  of  the  Former  Erie  Army  Depot  is  presented  in  the 
ESTCP  Demonstration  Plan  for  the  site  (ESTCP,  2006)  and  Pope  et  al.  1996. 
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Table  2.  Performance  Objectives 


Type  of 

Performance  Objective 

Primary  Performance 
Criteria 

Expected  Performance 
(Metric) 

Primary/ Qualitative 

Ease  of  use  and  efficiency  of 
operations  for  each  sensor  system 

Efficiency  and  ease  of  use  meets  design 
specifications 

Primary/Quantitative 

Geo-reference  position  accuracy 

Horizontal  <  0.25  m; 

Vertical  <  0.5  m 

Secondary/ Quantitative 

Survey  coverage 

>0.95  of  planned  survey  area 

Secondary/Quantitative 

Operating  parameters  (altitude, 
speed,  overlap,  production  level) 

Altitude:  1-3  m  AGL; 

Speed:  15-20  m/s  (30-40  kts); 

Overlap:  10% 

Production  level:  300  acres/day 

Primary/Quantitative 

Noise  level  (combined 
sensor/platform  sources,  post- 
filtering) 

<1  nT 

Secondary/ Quantitative 

Data  density/point  spacing 

0.5  m  along  track 

1.5  m  cross  track 

Secondary/Quantitative 

Modeling  parameter  definition 
(X,  Y  location;  depth;  and  size) 

X,Y  Location  :  90%  within  0.5m; 

Depth:  90%  within  0.5  m; 

Size:  90%  in  correct  Small/Medium/Large 
classification 
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Several  previous  activities  have  uncovered  a  variety  of  munitions  types.  These  activities  have 
included  dredging,  a  Time  Critical  Removal  Action  (TCRA)  and  a  beach  removal  action.  The 
munitions  types  are  listed  in  Table  3.  In  addition  to  those  known  to  have  been  fired  over  the  life 
of  the  range,  it  has  been  anecdotally  reported  (but  unconfirmed)  that  munitions  were  dumped 
from  barges  in  the  vicinity  of  the  impact  area  during  the  1960s.  The  types  of  munitions  and 
quantities  were  not  documented. 


Table  3.  Compilation  of  Munitions  Found  in  Previous  Activities 


Munition  Type 

Comments 

3.5  inch  rocket 

Found  during  dredging 

60  mm  mortar 

Found  during  dredging  and  beach  removal 
action  and  TCRA 

106  mm  projectile 

Found  during  dredging  and  beach  removal 
action  and  TCRA 

M52  fuze 

Found  during  dredging 

Ml 5  Smoke  Grenade 

Found  during  dredging 

105  mm  projectile 

Found  during  dredging  and  beach  removal 
action  and  TCRA 

90  mm  projectile 

Found  during  dredging  and  TCRA 

20  mm  projectile 

Found  during  Beach  Removal  Action 

165  mm 

Pieces  found  during  Beach  Removal  Action 

40  mm 

Found  during  TCRA 

75  mm 

Found  during  TCRA 

81  mm 

Found  during  TCRA 

155  mm 

Found  during  TCRA 

A  number  of  impact  areas  in  Lake  Erie  were  established  in  order  to  conduct  test  firings. 
Approximately  96,000  acres  within  Lake  Erie  and  1,427.75  acres  of  adjacent  lands  (wetlands, 
beach  and  dry  land)  are  classified  as  formerly  used  target  areas.  The  currently  maintained 
impact/safety  zone  used  by  Camp  Perry  includes  approximately  36,033  acres  of  the  FUDS  Lake 
impact  zone  (ESTCP,  2006). 

The  former  Erie  Army  Depot  was  formerly  used  for  artillery  testing,  resulting  in  impact  areas  on 
land,  in  Lake  Erie,  and  in  the  channel,  marshlands  and  beaches  of  the  Toussaint  River  where  it 
fronts  the  former  Depot.  The  designated  study  site  for  the  HeliMag  demonstration  encompasses 
3,300  acres  (see  Figure  1)  of  the  overall  50,000  acre  area.  This  study  site  includes  the  FUDS 
beach  from  just  northwest  of  the  Toussaint  River,  south  to  Camp  Perry,  and  the  beaches  and 
near-shore  areas  at  the  mouth  of  the  Toussaint  River. 
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The  physiographic  character  and  known  munitions-use  history  of  the  study  area  are  discussed  in 
some  detail  in  the  ESTCP  Demonstration  Plan  for  this  site  (ESTCP,  2006)  and  in  the  technical 
report  of  previous  MEC  assessments  at  the  site  (Pope  et  ah,  1996).  The  site  characteristics  most 
relevant  to  the  HeliMag  technology  demonstration  are  described  briefly  below. 

Topography  and  Soils.  The  study  area  is  located  along  the  south  shore  of  the  western  basin  of 
Lake  Erie.  The  land  is  a  low,  flat,  broad  plain  of  sands  deposited  on  top  of  older  lake  clays.  The 
beach  is  a  narrow,  shallow  depth,  sandy  barrier  which  includes  wash-over  deposits  and  evidence 
of  breaching.  The  FUDS  shore  has  a  history  of  rapid  erosion  and  retreat.  Rubble-mound 
revetments  have  been  installed  as  shore  protection  at  the  southeastern  end  of  the  study  site 
through  the  Camp  Perry  boundary  and  fronting  approximately  0.8  km  (0.5  mile)  of  the  central 
beach  (Figure  5).  Beach  width  varies  from  no  dry  beach  zone  up  to  approximately  150  m  (500 
feet). 

Terrestrial  Vegetation.  The  back  beach  area  consists  of  areas  of  thickly  vegetated  marshlands 
within  an  open  water  channel  and  lagoon  complex;  scrub  and  woodlands  exist  in  many  areas  as 
well.  The  trees  extend  in  many  areas  close  to  the  water  line,  which  limited  the  shoreline  area  for 
helicopter  surveys  (Figure  6). 

Aquatic  Environment.  Water  levels  in  the  study  site  respond  to  normal  annual  fluctuations  in 
Lake  Erie.  The  ESTCP  Demonstration  Plan  notes  that  the  western  end  of  Lake  Erie  is  shallow 
and  subject  to  rapid  water  level  fluctuations  as  storms  and  frontal  passages  can  seiche  (form  a 
standing  wave)  both  the  local  and  entire  lake  water  surface.  Lake  Erie  is  particularly  prone  to 
wind-caused  seiches  due  to  its  elongated  shape  and  shallowness.  The  survey  area  is  comprised  of 
shallow  water  areas  only  (limited  to  a  few  feet  in  depth).  Storms  causing  wind-caused  seiches 
would  preclude  surveys  of  the  near  shore  areas  due  to  fluctuation  in  water  levels  and  winds. 

Climate  and  Hydrology.  Changeable  weather  conditions  are  common  in  the  Lake  Erie  region, 
especially  in  the  spring  and  fall.  The  Great  Lakes  have  an  influence  the  climate  of  the  region  by 
acting  as  a  heat  sink.  The  lakes  moderate  the  temperatures  of  the  surrounding  land,  cooling  the 
summers  and  warming  the  winters.  The  lakes  also  act  as  a  giant  humidifier,  increasing  the 
moisture  content  of  the  air  throughout  the  year.  In  the  winter,  this  moisture  condenses  as  snow 
when  it  reaches  the  land,  creating  heavy  “lake  effect”  snowfall  on  the  downwind  shores  of  the 
lakes  in  some  areas.  Ice  frequently  covers  Lake  Erie.  However,  since  demonstration  was 
conducted  mid- August,  it  was  not  expected  that  climate  would  limit  the  ability  of  the  technology 
to  survey  the  area. 

Land  Use.  Multiple  land  uses  occur  within  and  near  the  study  site.  Along  the  northwest  shore  of 
the  Toussaint  River  is  a  section  of  the  Navarre  Division  of  the  Ottawa  National  Wildlife  Refuge 
and  the  Davis-Besse  Nuclear  Power  Station.  The  power  station  required  aircraft  information  for 
the  survey  areas  in  its  vicinity.  In  addition,  a  privately  owned  hunt  club,  The  Toussaint  Shooting 
Club,  is  located  in  the  northwest  portion  of  the  marshlands.  Last,  the  range  fan  for  Camp  Perry 
extends  into  the  survey  area;  Camp  Perry  agreed  to  allow  surveying  in  this  area  from  September 
1 1th  to  the  15  th.  The  southwestern  portion  of  the  marsh  contains  a  portion  of  the  former  Erie 
Army  Depot,  which  is  now  the  Erie  Industrial  Park;  most  of  the  buildings  in  the  industrial  park 
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are  used  for  light  commercial  and  storage  purposes.  However,  the  most  lake-ward  complex, 
which  incorporates  the  original  Erie  Army  Depot  firing  bunkers,  is  owned  by  ARES,  Inc.,  which 
operates  a  commercial  artillery  and  armament  test  facility.  Both  ARES  and  Camp  Perry  continue 
to  use  portions  of  the  FUDS  (both  lake  and  marsh)  for  ordnance  testing  purposes. 


Figure  5.  Erosion  prevention  measures  along  shoreline. 


Figure  6.  Terrestrial  vegetation. 
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3.4.  Pre-Demonstration  Testing  and  Analysis 

Helicopter  magnetometry  technology  has  been  fully  evaluated  by  DoD  through  ESTCP.  As  a 
WAA  tool,  the  NRL  MTADS  has  been  previously  demonstrated  at  other  WAA-PP  sites, 
including:  Kirtland  Precision  Bombing  Range,  New  Mexico;  Victorville  Precision  Bombing 
Range,  California;  and  Pueblo  Precision  Bombing  Range,  Colorado.  Additionally,  Sky  Research 
has  begun  deploying  the  updated  helicopter-borne  magnetometry  sensor  system  to  U.S.  Air 
Force  sites  under  the  Air  Force’s  Military  Munitions  Response  Program.  To  date,  HeliMag  data 
have  been  collected  at  Edwards  Air  Force  Base,  California,  with  four  more  Air  Force  sites 
scheduled  throughout  2007. 

Because  the  innovations  by  Sky  Research  changed  both  the  platform  and  some  of  the  technology 
components,  the  new  HeliMag  system  was  compared  to  the  NRL  MTADS  system  deployed  on  a 
Bell  Long  Ranger  and  previously  demonstrated  for  the  ESTCP  WAA-PP  before  the  new  system 
was  deployed  to  the  Toussaint  River  site.  The  two  systems  have  never  been  deployed  over  the 
same  area.  Therefore,  data  collected  during  high  altitude  flights  in  different  areas  and  on 
different  days  were  used  to  perform  the  comparison.  In  both  cases,  a  20  second  section  of  data 
for  comprehensive  analysis  was  selected  for  the  system  comparison.  The  analysis  showed  that 
the  Hughes  /Sky  DAS  combination  had  a  lower  noise  floor  than  the  Bell/MTADS  combination 
The  side-by-side  comparison  of  the  results  of  calibration  lane  flights  demonstrating  the  two 
system’s  capabilities  was  provided  to  ESTCP  prior  to  mobilization  (Appendix  A). 

Table  4  below  summarizes  the  high  altitude  comparisons.  Note  that  sensor  7  for  the  Sky  system 
was  malfunctioning  and  the  noise  floor  of  that  sensor  is  not  reported. 


Table  4.  Standard  Deviations  of  the  Bell  Helicopter  with  NRL  Data  Acquisition  Computer 

and  the  Hughes  Helicopter  with  Sky  DAS 


Raw 

NRL 

Processed 

NRL 

Raw 

Sky 

Processed 

Sky 

1 

1.11 

0.13 

0.40 

0.07 

2 

1.48 

0.14 

0.60 

0.09 

3 

1.52 

0.18 

0.80 

0.11 

4 

1.26 

0.22 

0.80 

0.11 

5 

1.36 

0.10 

0.57 

0.09 

6 

0.94 

0.07 

0.34 

0.07 

7 

0.54 

0.06 

N/A 

NA 
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3.5.  Testing  and  Evaluation  Plan 

3.5.1.  Demonstration  Set-Up  and  Start-Up 

Mobilization  for  this  project  required: 

1)  Mobilization  of  the  equipment,  pilot,  and  sensor  operators. 

2)  Deployment  of  ground  support  personnel  to  establish  ground  fiducials,  establish  and 
operate  GPS  base  stations,  establish  calibration  line  location  and  collect  data  on 
calibration  location,  and  provide  logistical  support. 

3)  Establishment  of  calibration  line  and  standard  pre-collection  maintenance  and  calibration 
procedures  established  during  previous  deployments. 

A  base  of  field  operations  was  established  at  the  Carl  R.  Keller  Field,  Port  Clinton,  OH,  to 
provide  fuel  and  temporary  hanger/storage  space  during  operations  at  the  site.  The  methods  to 
establish  base  station  survey  monuments,  emplaced  calibration  items  and  ground  targets 
appropriate  for  the  sensors  have  been  described  in  the  demonstration  plan  for  the  former  Pueblo 
Precision  Bombing  Range  (Foley,  2005). 

Ground  Control 

RTK  GPS  provided  cm-accuracy  real-time  positioning  and  was  used  with  the  HeliMag  system.  It 
was  also  used  to  generate  positions  for  ground  fiducials  and  for  positioning  ground  calibration 
data  and  field  verifications.  Sky  Research  employs  an  in-house  professional  land  surveyor  to 
assure  that  geospatial  data  generated  by  the  project  maintain  accurate  ties  to  the  local  coordinate 
system  and  to  oversee  the  accurate  field  emplacement  of  fiducials  for  data  registration  and 
surrogate  targets  for  sensor  calibration  and  verification  of  classification  and  analysis  algorithms. 

Sensor  Calibration  Targets 

A  calibration  line  was  established  at  the  base  of  field  operations  at  the  airport.  To  confirm  its 
suitability  for  the  calibration  line  placement,  background  noise  data  was  collected  and  reviewed 
before  the  calibration  line  was  established.  The  calibration  targets  were  placed  on  the  ground 
surface  at  a  spacing  of  50  m  at  the  orientations  as  listed  in  Table  5.  Target  positions  were 
surveyed  at  two  altitudes  and  the  resulting  signatures  compared  to  calculated  responses  to 
confirm  that  the  system  is  operating  at  its  expected  sensitivity.  Calibration  line  surveys  were 
conducted  twice  each  day.  No  targets  were  buried  and  no  attempt  was  made  to  measure  a 
probability  of  detection. 
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Table  5.  Emplaced  Calibration  Targets 


Item 

Depth 

Orientation 

8”  steel  cube 

Ground  level 

100  lb.  bomb  stimulant 

Ground  level 

1  N-S 

1  E-W 

155  mm  projectile 

Ground  level 

1  N-S 

1  E-W 

2.75”  warhead 

Ground  level 

1  N-S 

1  E-W 

3.5.2.  Period  of  Operation 

Pre-planning  for  the  survey  was  conducted  in  August  of  2006,  including  submittal  of  the 
demonstration  plan  and  final  acceptance  by  the  ESTCP  Program  Office.  The  ground  surveys 
were  conducted  in  August  prior  to  mobilization  of  the  ground  crew  and  helicopter  to  the  survey 
site.  Mobilization  of  the  helicopter  from  Ashland,  Oregon,  to  Ohio  began  on  September  5th,  and 
the  helicopter  arrived  on  site  on  September  7th.  The  ground  crew  mobilized  from  Denver, 
Colorado,  on  September  6th  and  transported  the  fuel  truck  and  equipment,  arriving  on  site  on 
September  7th.  The  sensor  boom  assembly,  calibration  lane  construction  and  test  flights  were 
conducted  on  September  8th. 

Surveys  began  on  September  9th.  The  initial  area  surveyed  was  within  the  Camp  Perry  exclusion 
area  because  of  the  Camp  Perry  cooperation  in  suspending  activities  in  this  area  for  one  week; 
re-flies  were  conducted  in  this  area  on  September  15th  to  ensure  data  quality  for  the  exclusion 
area.  Surveying  continued  through  the  morning  of  September  16th;  on  the  morning  of  the  16th, 
the  survey  ended  when  pilot  error  caused  an  accident  which  destroyed  the  helicopter,  booms,  and 
much  of  the  electronic  equipment.  While  the  helicopter  suffered  significant  structural  damage, 
the  pilot  and  sensor  operator  escaped  without  injury.  A  summary  of  the  acres  per  day 
productivity  prior  to  the  16th  is  provided  in  Table  6. 

Table  6.  HeliMag  Data  Collection  Acreage 


Data  Collection  Day 

Acres  Surveyed 

September  9,  2006 

400 

September  10,  2006 

500 

September  11,  2006 

517 

September  12,  2006 

494 

September  13,  2006 

708 

September  14,  2006 

659 

September  15,  2006 

111 

Acres  Collected 

3,389  acres 

Average  Daily  Productivity 

484  acres/day 
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3.5.3.  Operating  Parameters  for  the  Technology 

Sky  Research  deployed  the  airborne  MTADS  system  on  the  MD530  helicopter  platform, 
together  with  pilot,  system  operator  and  ground  support  team  to  operate  the  RTK  GPS  base 
stations.  The  system  uses  an  array  of  seven  full-field  Cs  vapor  magnetometers  deployed  on  a  9  m 
boom  mounted  transversely  on  the  front  of  the  helicopter  skids.  The  helicopter  is  typically  flown 
at  a  low  altitude  (1.5-5  m),  with  a  forward  velocity  of  15  -  20  m/s.  The  sensor  boom  extends 
well  in  front  of  the  helicopter  and  is  clearly  visible  to  the  pilot,  which  is  important  for  these  very 
low  flight  altitudes. 

With  the  sensor  spacing  of  1.5  m,  a  data  collection  rate  of  100  Hz,  and  a  speed  over  ground  of  20 
m/s,  the  resulting  data  density  provides  50  data  points  on  a  typical  target  to  fit  the  dipole 
signature.  Magnitude  data  points  are  merged  with  GPS  positioning  data  to  generate  geo- 
referenced  point  data  that  are  interpolated  into  magnetic  surface  maps,  and  analyzed  for  target 
picking  and  characterization. 

Airborne  MTADS 

•  7  Cs  vapor  full-field  magnetometers 

•  1.5  m  spacing  on  9  m  boom 

•  along-track  data  density:  15-20  cm  typical 
Ground  Support 

•  RTK  GPS  base  stations  w/  radio  link 
Positioning  and  geo-referencing 

.  RTK  GPS 
Processing 

•  Custom  DAS  merging  and  point  file  creation 

•  UXOLab  for  initial  surface  interpolation 

•  UXOLab  for  target  picking  and  classification 

•  ArcGIS  for  final  interpolation  and  mosaicing,  mapping,  presentation 

•  ArcSDE/Sky  Research  Geophysical  Data  Center  for  storage  and  management 

Flight  Speed 

.  15-20  m/s  (30-40  kts) 

Altitude 

.  1.5-3  m  AGL 

Spatial  Accuracy 

•  <  5  cm  RMSE 
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3.5.4.  Data  Processing 

Data  processing  was  completed  nightly  following  data  collection  survey.  Data  were  downloaded 
via  zip  disks  and  uploaded  via  the  Internet  after  each  survey  mission.  Data  processing  was 
performed  using  custom  application  software  running  under  the  UXOLab  geophysical  data 
processing  environment.  An  overview  of  this  process  is  outlined  in  the  flow  chart  below  (Figure 

7). 

Flight  x  Raw  Data  Files 

‘’ill! 


Survey  ‘Master’  Mag  Database  Survey  ‘Master’  DEM  Database 


•Filter  magnetic  data  •Filter  and  calibrate  acoustic  altimeter  data 

•Remove  geologic  background  'Filter  laser  altimeter  data 


Figure  7.  Helicopter  MTADS  processing  flow  chart. 


During  the  first  data  processing  stage,  the  raw  data  for  a  given  survey  flight  were  time-aligned 
and  transcribed  from  the  various  raw  data  files  into  a  ‘flight’  database.  Routines  were  run  to 
automatically  reject  or  ‘default’  invalid  data.  Data  were  rejected  based  upon  status  flags  present 
in  the  raw  data  records  or,  in  the  case  of  the  magnetometer  data,  a  simple  ‘in  range’  test  may  be 
used.  The  GPS  geographic  position  coordinates  were  transformed  to  WGS84  UTM  coordinates. 
At  this  point  the  data  were  visually  inspected  to  ensure  both  integrity  and  quality.  This  pre¬ 
processing  stage  is  instrumentation-specific  and  the  steps  required  to  transcribe  these  data  into  a 
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time-aligned  database  were  dictated  by  the  structure  of  the  data  outputs  from  each  device  and  the 
manner  in  which  they  were  logged.  All  data  outputs  were  received  by  the  on-board  DAS.  A  time 
stamp  was  appended  to  each  sample  data  string  and  the  sample  was  then  stored  in  a  separate  data 
file  for  each  device.  Table  7  provides  a  list  of  the  raw  data  input  files  generated. 


Table  7.  Helicopter  MTADS  Raw  Data  Input  Files 


Device 

Sample 

Rate 

(Hz) 

Data 

Type 

Filename  extension 

Remarks 

Geometries 

Model  822A  Cs 
Magnetometers 

400 

RS232- 

ASCII 

M 1 9yycldcl  1  A. sen 

M 1 9mM/2A.sen 

M 1 9yyddcB  A .  s  en . . . 

A  separate  file  is  created  for 
each  of  7  magnetometers.  The 

400  Hz  data  are  low-pass  filtered 
and  de-samp  led  to  100  Hz. 

Trimble  Model 
MS750  GPS 
position/attitude 
data 

20/10 

RS232- 

ASCII 

P 1 3yydddA.pos 

Position  data  are  in  Trimble 

GGK  message  format,  azimuth 
and  roll  are  in  Trimble  AVR 
message  format. 

Trimble  Model 
MS750  GPS 
time  tag 

1 

RS232- 

ASCII 

T 1 4yydddA.time 

Used  to  resolve  the  integer 
ambiguity  of  the  GPS  PPS  signal 
in  real-time  -  only  stored  for 
diagnostic  purposes. 

Optech  Model  60 
Laser  Altimeter 

10 

RS232- 

ASCII 

A1 5>y<7JdA.alt 

Measures  helicopter  height 

AGL. 

Acoustic 

altimeters 

10 

Analog 

voltage 

D22yyddd\ A.dst 

Measures  sensor  array  height 

AGL  at  two  points. 

An  important  consideration  for  integration  of  the  positioning  system  with  geophysical  sensors  is 
that  of  time  alignment.  For  dynamic  applications,  the  time  of  applicability  (TOA)  of  the 
geophysical  sensor  data  must  be  aligned  with  the  TOA  of  the  measured  positioning  data  to 
within  one  millisecond.  Any  measurement  will  have  some  latency  before  the  data  are  collected 
and  stored,  which  may  be  static  or  variable  in  nature.  In  addition  to  this  latency,  conventional 
time  stamping  of  RS232  data  is  not  precise  and  can  inject  hundreds  of  milliseconds  of  additional 
delays.  Thus,  simply  time  stamping  the  positioning  data  as  it  is  transmitted  to  the  DAS  does  not 
ensure  that  the  TOA  of  the  positions  can  be  precisely  aligned  with  that  of  the  geophysical  data. 

GPS  systems  commonly  have  an  internal  latency  that  is  variable  (i.e.,  the  time  between  the 
applicability  of  a  given  measurement  and  the  transmission  of  the  derived  position  will  vary)  in 
addition  to  the  serial  port  variability.  To  allow  users  to  know  precisely  when  a  measurement 
applies,  the  data  message  is  time  stamped  (i.e.,  the  position  solution  is  given  in  four  dimensions; 
time,  X,  Y,  and  Z)  to  a  very  high  degree  of  precision.  This  time-stamp  is  the  precise  TOA  of  the 
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GPS  position  and  attitude  data.  In  addition,  GPS  receivers  also  output  a  pulse  per  second  (PPS) 
trigger  at  every  precise  integer  second  to  provide  a  means  to  synchronize  the  DAS  time  with 
GPS  time.  The  SKY  DAS  uses  this  timing  pulse  and  the  Universal  Time  Coordinated  (UTC) 
time  message  from  the  Trimble  receiver  to  align  its  internal  clock  to  GPS  time  with  a  very  high 
degree  of  precision.  The  counting  of  the  magnetometer  Larmor  signals  is  performed  by  a 
counter-board  that  is  fully  integrated  into  the  SKY  DAS.  This  allows  precise  alignment  of  the 
TOA  of  the  magnetometer  data  with  respect  to  the  DAS  time  base. 

The  processing  steps  used  to  derive  a  final  geo-referenced  data  set  and  associated  images  were  as 
follows: 

1)  The  raw  data  are  collected  at  a  sample  rate  of  400  Hz.  After  application  of  a  low-pass 
filter  the  data  are  de-sampled  to  100  Hz.  Each  de-sampled  magnetometer  value  and 
associated  time  stamp  (UTC)  are  transcribed  into  a  Geosoft  database  into  appropriately 
named  channels  (e.g.  Time,  Magl  raw,  Mag2_raw,  etc.). 

2)  Using  the  existing  Time  channel  and  the  UTC  time  stamp  in  each  GPS  data  record,  the 
GPS  position  and  orientation  data  are  interpolated  to  each  magnetometer  record.  As  part 
of  this  process,  the  GPS  master  antenna  positions  are  translated  to  that  of  each  sensor 
based  upon  the  aircraft  geometry  and  attitude. 

3)  The  DAS  time  channel  and  the  DAS  time  field  in  the  raw  data  files  are  used  to 
interpolate  the  ancillary  data  for  each  magnetometer  record.  The  ancillary  data  channels 
include  the  following:  laser,  four  acoustic  altimeter  channels  (two  for  each  acoustic 
altimeter  station  to  provide  redundancy),  and  fluxgate  X,  Y,  and  Z  components. 

4)  After  the  data  are  transcribed  into  a  database,  invalid  data  are  defaulted  to  ‘dummy’ 
values.  The  magnetometer  data  are  defaulted  outside  of  a  reasonable  range  and  the  GPS 
data  are  defaulted  based  upon  the  values  of  the  two  status  flags. 

5)  Long  wavelength  magnetic  features  are  removed  from  the  data  through  the  use  of  a  de- 
trend  filter.  This  filter  combines  an  iterative  de-median  process  with  a  running  average  to 
derive  the  long-wavelength  components  that  are  removed  from  the  magnetic  data.  The 
filter  parameters  are  dependent  upon  the  local  site  geology  and  the  aircraft  survey 
altitude. 

6)  The  final  filtered,  geo-referenced,  magnetic  data  are  used  to  create  Geosoft  grids  using  a 
1  grid  cell  size.  These  grids  are  used  as  the  basis  for  the  target  selection  process. 

3.5.5.  Demobilization 

The  survey  was  terminated  early  due  to  an  accident  which  destroyed  the  helicopter,  booms  and 
much  of  the  electronic  equipment.  An  effort  was  made  to  salvage  any  equipment  that  was  not 
totally  destroyed.  All  of  the  salvaged  equipment  and  wreckage  were  removed  from  the  site. 
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4.  Performance  Assessment 


4.1.  Data  Calibration  Results 
4.1.1.  Data  Calibration 


The  data  collected  over  each  target  from  the  calibration  line  passes  that  are  assumed  to  be  valid 
(i.e.,  target  positions  are  stable  and  data  positioning  quality  is  good)  were  analyzed  with  the  Sky 
Research  UXOLab  dipole  fit  algorithm.  This  analysis  derives  the  parameters  for  a  model  dipole 
that  best  fits  the  observed  data.  These  parameters  include  horizontal  position,  depth,  size,  and 
solid  angle  (i.e.,  the  angle  between  the  Earth’s  magnetic  field  vector  and  that  of  the  dipole 
model).  The  derived  parameters  were  examined  for  accuracy,  (determined  as  the  average  error  or 
‘bias’  where  relevant),  and  repeatability  (indicated  by  the  standard  deviation),  presented  in  Table 
8. 


Table  8.  Calibration  Results  for  Calibration  Lane  Targets 


Dipole  Fit  Parameter 

Bias 

Standard  Deviation 

Easting 

-0.17  m 

0.11  m 

Northing 

-0.19  m 

0.07  m 

Depth 

0.10m 

0.19  m 

Size 

NA 

0.012  m 

Solid  Angle 

NA 

3.7° 

Figure  8  shows  the  derived  positions  for  each  target  relative  to  the  ground  truth  supplied.  The 
accuracy  of  these  positions  relative  to  the  ground  truth  is  well  within  the  range  expected  for  the 
MTADS  system.  The  calibration  lines  were  flown  in  an  N-S  direction,  resulting  in  better 
precision  in  the  Northing  than  in  the  Easting  due  to  the  difference  in  sample  density  (along-track 
sample  density  is  5  to  10  times  higher  than  for  across-track).  This  is  consistent  with  our  findings 
from  earlier  projects  (Foley  and  Wright  2006).  A  significant  component  of  the  bias  reported  in 
Table  8  and  reflected  in  the  ‘Position  Error’  data  in  Figure  8  (blue  diamonds)  is  probably  due  to 
errors  in  the  ground  truth.  The  ground  truth  positions  were  not  surveyed  in  after  emplacement, 
due  to  the  unexpected  nature  of  the  conclusion  of  the  data  collection  (the  reacquisition  of  ground 
truth  position  data  was  simply  overlooked  in  the  confusion  related  to  salvage  of  the  helicopter 
and  equipment).  The  coordinates  used  for  analysis  were  the  planned  target  position  coordinates. 
Errors  of  a  few  tens  of  centimeters  would  be  expected  between  planned  emplacement  and  actual 
emplacement. 
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Figure  8.  Calibration  line  target  positioning  errors. 


4.1.2.  Calibration  Item  Response 

The  dipole  fit  size  estimate  for  any  given  munitions  item  will  vary  considerably  depending  upon 
the  alignment  of  the  object  with  the  Earth’s  magnetic  field.  Therefore  the  size  can  only  be  used 
as  a  coarse  estimate  of  the  object  size.  For  this  reason,  the  accuracy  of  the  size  estimate  of  the 
calibration  items  is  not  of  particular  import  when  discussing  the  system  performance,  other  than 
simply  verifying  that  the  estimate  falls  within  the  expected  range  for  a  given  target.  The  size 
results  for  like  items  are  relatively  consistent  with  the  exception  of  the  100  lb  bombs.  It  appears 
that  the  difference  in  orientation  of  these  two  targets  with  respect  to  the  earth’s  field  has  a  large 
impact  on  the  derived  size  of  these  targets.  This  finding  is  not  inconsistent  with  the  results  shown 
in  Billings  et  al.  (2004).  Because  the  calibration  data  consist  of  repeated  flights  over  the  same 
stationary  targets,  the  repeatability  of  the  derived  size  estimates  demonstrates  consistency  in 
system  performance  (Figure  9). 

In  a  manner  similar  to  the  size  estimates  discussed  above,  the  dipole  fit  solid  angle  estimates 
depend  heavily  on  the  orientation  of  the  target  relative  to  the  Earth’s  magnetic  field.  In  the  case 
of  the  calibration  line  test  targets,  the  ‘ground  truth’  is  unknown  and  not  really  important. 
However  the  stability  of  this  prediction  for  repeated  flights  over  the  calibration  line  is  indicative 
of  the  performance  of  the  airborne  system  (Figure  10). 
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Figure  9.  Dipole  fit  size  estimates  for  calibration  line  targets. 
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Figure  10.  Dipole  fit  solid  angle  estimate  for  calibration  line  targets. 
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4.2.  Anomaly  Selection/ Analysis 

4.2.1.  Anomaly  Picking  Methodology 

For  the  purposes  of  WAA,  the  main  goal  is  to  delineate  target  density  throughout  the  survey  site. 
Target  selection  can  be  accomplished  either  manually  or  through  automated  routines;  the 
geologic  background  signal  largely  determines  what  methods  are  best  for  a  given  site.  Manual 
target  selection  is  both  subjective  and  labor  intensive.  The  results  obtained  will  vary  considerably 
depending  upon  the  skill  level  of  the  analyst;  even  an  experienced  analyst  will  find  it  difficult  to 
be  consistent  with  respect  to  his/her  ability  to  select  targets  that  are  masked  by  geologic  signal  or 
overlapping  signal  from  other  targets.  In  areas  of  “quiet”  geologic  background,  automatic  target 
pickers  can  be  faster  to  use,  scientifically  repeatable  and  more  objective  than  manual  target 
picking. 

Automatic  target  pickers  are  not  yet  sophisticated  enough  to  reliably  detect  closely  spaced  targets 
or  targets  that  are  at  or  below  the  same  amplitude  as  local  geologic  signal.  Where  a  reasonably 
experienced  analyst  is  able  to  successfully  discriminate  a  large  number  of  targets  from  localized 
geologic  signals  that  are  of  the  same  amplitude  or  higher,  the  automatic  target  detection  routines 
that  are  currently  available  are  not  able  to  differentiate  between  our  targets  of  interest  and  local 
geologic  anomalies.  As  a  result,  automatic  target  selection  routines  must  only  be  used  to  select 
targets  with  response  amplitudes  significantly  above  the  nominal  geologic  noise;  otherwise,  an 
inordinate  number  of  false  targets  are  selected.  Furthermore,  the  automatic  routines  do  not 
perform  well  in  areas  of  high  target  density. 

In  practice,  the  decision  to  pick  manually,  or  use  an  auto-picker  then  add/reject  targets  manually 
is  made  based  upon  the  number  of  targets  to  be  picked  and  the  extent  of  geo logic/anthropo logic 
clutter  that  must  be  dealt  with.  On  this  project  there  were  relatively  few  anomalies  and  a 
significant  number  of  anthropologic  features.  For  this  reason  the  decision  was  made  to  pick  the 
targets  manually. 

At  the  Toussaint  River  site  the  distance  of  the  sensor  above  potential  targets  was  increased  by  the 
water  depth.  This  resulted  in  significant  reduction  of  anomaly  amplitudes  (particularly  in  deeper 
water)  and  a  corresponding  reduction  in  signal  to  noise  of  the  targets.  As  a  result,  manual  target 
selection  was  required  because  the  automatic  target  pickers  do  not  perform  well  on  low  signal  to 
noise  targets. 

4.2.2.  Anomaly  Picking  Results 

Using  manual  picking  procedures,  1,904  anomalies  were  selected  from  the  data  to  assess  the 
distribution  of  metal  objects  across  the  study  area.  Figure  1 1  illustrates  the  locations  of  these 
anomalies  and  their  densities  over  the  WAA  study  area. 
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Figure  11.  Target  anomalies  selected  within  the  survey  area  and  their  densities. 
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4.2.3.  Metal  Density  Analysis 

To  visualize  the  distribution  of  metal  objects  across  the  study  area,  a  density  raster  was  computed 
using  a  75  m  radius  neighborhood  kernel  that  assigned  anomaly  densities  in  anomalies  per 
hectare  to  each  cell  in  the  raster.  Simply  described,  at  grid  nodes  of  every  two  meters  the  number 
of  targets  that  appear  within  a  100  m  search  radius  were  counted.  This  search  radius  provides  the 
density  in  targets  per  31,416  m2.  These  values  were  then  ‘normalized’  by  diving  by  3.1416  to 
provide  density  estimates  in  targets/hectare.  The  resulting  data  were  gridded  to  provide  anomaly 
density  images. 


Our  ability  to  detect  objects  of  a  given  size  is  dependent  upon  our  survey  altitude  above  the 
object.  In  terrestrial  surveys  the  measured  survey  altitude  can  be  used  to  support  assumptions 
about  the  size/depth  of  detectable  targets.  For  example,  based  upon  Figure  12  we  expect  to  see 
all  155  projectiles  (no  remanent  magnetic  signal)  down  to  a  depth  of  2.5  m.  However,  when 
flying  over  water  we  must  be  cognizant  of  the  fact  that  the  survey  altitude  above  the  lake  bottom 
is  not  known.  Therefore  any  assumptions  regarding  the  minimum  size  of  detectable  targets  can 
only  be  crudely  estimated. 


Minimum  Detectable  Size  as  a  Function  of 
Sensor  Offset  Distance 
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Figure  12.  Minimum  detectable  ordnance  size  as  a  function  of  the  separation  distance 
between  the  sensor  and  the  target.  This  distance  is  the  sum  of  the  sensor  altitude  above 
water,  water  depth  and  burial  depth  below  the  lake  bed. 
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As  a  result  of  the  increasing  water  depth,  the  target  density  estimates  are  skewed  to  show  lower 
target  densities  as  the  water  gets  deeper.  The  relative  densities  moving  along  the  shore  line  or, 
more  accurately,  along  lines  of  equivalent  depths  will  be  valid.  Additionally,  we  can  assume  that 
the  predicted  density  of  targets  as  large  as  155  mm  targets  will  be  valid  to  water  depths  of 


1  -2  m. 


4.2.4.  Target  Dipole-Fit  Analyses 

Each  selected  anomaly  was  analyzed  using  the  same  dipole  fit  analysis  described  in  Section 
4.1.1.  The  fit  results  are  provided  in  a  spreadsheet  called  “Toussaint_R_Dipole_Fit_Results.xls”. 
Of  the  1,904  selected  targets,  1,155  were  successfully  fit  to  a  dipole  model.  The  lack  of  success 
in  fitting  the  remaining  targets  is  attributed  to  low  signal  to  noise  for  these  targets.  The 
distribution  of  size  estimates  derived  from  these  analyses  is  shown  in  Figure  13.  There  is  a 
conspicuous  absence  of  targets  smaller  than  150  mm.  This  is  attributed  to  the  increased  target 
stand-off  distance  relative  to  previous  terrestrial  airborne  surveys.  We  assume  that  the  smaller 
targets  had  insufficient  signal  to  be  successfully  fit  to  a  dipole  model. 
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Figure  13.  Distribution  of  dipole  fit  size  estimates. 
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4.2.5.  Intrusive  Investigation  Results 

A  small  number  of  targets  were  selected  for  remediation  to  supply  ground  truth  that  could  be 
used  to  ‘calibrate’  the  results  of  the  dipole  fit  analyses.  These  results  are  presented  in  tabular 
format  in  Appendix  B  and  include  18  targets  selected  for  investigation,  summarized  as  follows: 

•  7  x  155  mm  projectiles 

•  2  x  2.75  inch  rockets 

•  4  x  non-UXO  related  objects 

•  2  x  objects  buried  in  the  lake  bottom  too  deep  to  identify 

•  3  x  ‘no-contact’ 

The  three  ‘no-contacts’  were  all  located  in  very  shallow  water  (0.25  m  or  less)  making  them 
susceptible  to  being  moved  by  wave  action  or  people  in  the  time  between  the  survey  and  the 
intrusive  investigation.  Of  the  seven  155  mm  targets,  only  two  were  fit  to  a  size  that  we  would 
expect  for  this  type  of  ordnance.  The  remaining  five  were  all  fit  to  a  size  that  is  significantly 
greater  than  we  would  expect.  We  attribute  this  phenomenon  to  the  presence  of  significant 
remanent  magnetization  of  these  targets.  In  Figure  14  we  present  the  recovered  moments  for 
these  targets.  Targets  1,  4,  5  and  6  exhibit  significant  rotation  of  their  moments  relative  to  the 
Earth’s  field.  This  and  the  variability  on  the  magnitude  of  these  moments  support  the  contention 
that  a  large  part  of  the  magnetic  response  of  these  targets  is  due  to  remanent  magnetization.  This 
evidence  is  consistent  with  earlier  unverified  reports  of  ordnance  being  put  into  the  lake  by 
means  other  than  test  firing. 


Figure  14.  Recovered  moments  of  155  mm  targets  identified  during  the  intrusive 
investigation.  The  moment  vectors  are  plotted  as  their  parallel  and  orthogonal  components 
relative  to  the  Earth’s  field.  The  black  circles  represent  expected  results  for  two  types  of 
155  mm  projectiles. 
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If  we  factor  in  a  significantly  increased  expected  size  for  155  mm  projectiles,  we  can  use  the  fit 
size  distribution  presented  in  Section  4.2.4  to  assume  that  a  very  large  percentage  of  the  targets 
detected  represent  ordnance.  This  is  also  supported  by  the  results  of  the  intrusive  investigation. 

4.3.  Performance  Criteria 

The  performance  of  the  helicopter  magnetometry  technology  was  measured  against  the  criteria 
listed  in  Table  9. 


Table  9.  Performance  Criteria  for  the  Toussaint  River  HeliMag  Technology  Demonstration 


Performance  Criteria 

Description 

Type  of  Performance 
Objective 

Technology  Usage 

Ease  of  use  and  efficiency  of  operations. 

Primary/ 

Qualitative 

Geo-reference  position 
accuracy 

Comparison  of  calibration  target  dipole  fit 
analysis  position  estimates  (in  3  dimensions)  to 
ground  truth. 

Primary/ 

Quantitative 

HeliMag  survey  area 
coverage 

Percentage  of  survey  area  coverage  surveyed. 

Secondary/  Quantitative 

Operating  parameters 
(altitude,  speed,  overlap, 
production  level) 

Valued  to  be  calculated  and  using  average  and 
mean  statistical  methods  to  compute  each 
parameter. 

Secondary/  Quantitative 

System  Noise 

Accumulation  of  noise  from  sensors  and  sensor 
platforms,  including  GPS,  rotor  noise,  radio 
frequencies,  etc.  calculated  as  the  standard 
deviation  of  a  20  second  window  of  processed 
data  collected  out  of  ground  effect. 

Primary/ 

Quantitative 

Data  density/point 
spacing. 

(#  of  sensor  readings/second)/  airspeed 

Secondary/ 

Quantitative 

MEC  parameter  estimates 

The  size  and  dipole  angle  estimates  of  the 
calibration  items  are  consistent. 

Secondary/ 

Quantitative 

4.4.  Performance  Confirmation  Methods 

Table  10  details  the  confirmation  methods  that  were  used  for  each  criterion,  the  expected 
performance  and  the  performance  achieved. 

In  light  of  the  helicopter  accident,  claiming  ‘pass’  for  the  ease  of  use  criteria  requires  some 
explanation.  It  is  Sky  Research’s  contention  that  the  accident  is  an  isolated  event  and  should  not 
be  used  in  our  evaluation  of  the  ease  of  use  criteria.  This  contention  does  not  deny  the 
seriousness  of  this  incident  nor  the  imperative  requirement  to  take  any  and  all  reasonable 
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measures  to  prevent  a  recurrence  of  this  type  of  incident.  However  in  the  context  of  this  report, 
we  do  not  feel  it  is  appropriate  to  factor  this  incident  into  our  performance  assessment. 

Position  accuracy  on  a  dynamic  platform  is  very  difficult  to  measure  precisely.  We  are  able  to 
infer  the  position  accuracy  of  the  sensor  data  by  using  the  position  estimates  derived  from  dipole 
fit  analysis  of  data  collected  over  known  targets.  Although  there  are  additional  error  sources 
(other  than  just  those  due  to  the  data  positioning)  in  the  dipole  fit  results,  they  are  almost 
negligible  due  to  the  stability  of  the  magnetometer  calibration  and  the  robustness  of  the  dipole  fit 
process.  Because  reciprocal  passes  will  tend  to  hide  along-track  position  errors  (due  to  the 
robustness  of  the  dipole  fit  process),  the  dipole  fit  analyses  were  performed  on  a  single  pass  over 
the  targets.  The  results  for  these  analyses  are  presented  in  Table  8. 

The  spatial  extent  of  a  magnetic  anomaly  (from  our  targets  of  interest)  is  a  factor  of  two  times 
greater  than  the  sensor  offset  distance.  Based  upon  our  minimum  survey  height  of  1.5  m,  we  can 
conservatively  define  gaps  in  survey  coverage  as  areas  where  the  distance  to  the  nearest  sensor 
reading  is  greater  than  2  m.  Gaps  in  survey  coverage  are  generally  related  to  navigation  (a 
combination  of  pilot  skill,  topography/vegetation,  and  wind  conditions)  or  data  integrity 
(primarily  GPS  fix  quality).  As  a  general  practice,  images  representing  the  data  from  each  day  of 
survey  flying  are  created  to  identify  areas  requiring  fill-in  flying  to  cover  significant  gaps  in 
coverage.  Invariably  there  will  be  a  number  of  gaps  in  survey  coverage  that  cannot  be  practically 
filled.  To  estimate  the  survey  coverage  performance,  at  every  0.25  m  interval  (grid  node)  we 
search  through  aim  radius  for  a  valid  data  point.  The  number  of  grid  nodes  where  valid  data  are 
found  is  divided  by  the  total  number  of  grid  nodes  to  derive  the  percentage  of  survey  coverage. 
Due  to  the  helicopter  accident,  a  portion  of  the  original  planned  survey  was  not  covered.  In 
addition,  re-flights  to  cover  significant  data  gaps  were  not  able  to  be  performed.  The  un-surveyed 
area  was  excluded  from  the  percent  coverage  calculation  but  the  gaps  where  re-flights  would 
normally  have  been  flown  were  included,  for  a  total  of  3,388  acres  covered  and  63.4  acres  not 
covered.  Based  upon  these  factors  and  acreages,  the  final  coverage  was  99.98%. 

The  assessment  of  the  survey  altitude  and  speed  was  performed  by  extracting  a  statistics  for 
these  parameters  from  the  survey  databases.  Survey  speed  was  consistently  maintained  between 
15  and  25  kts,  with  some  insignificant  variation  at  the  beginning  or  end  of  the  survey  lines. 
Survey  altitude  is  a  critical  parameter  for  this  type  of  investigation  and  is  expected  to  be  a  little 
more  variable  than  survey  speed.  In  Figure  15,  we  present  a  histogram  of  the  survey  altitude 
performance.  Prior  to  deriving  these  statistics,  all  altitudes  above  5  m  were  rejected.  These 
altitudes  occur  at  the  end  of  survey  lines  or  during  times  when  the  helicopter  has  broken  off  a 
survey  line  and  is  circling  back  to  reacquire  it.  The  mean  survey  altitude  was  2.0  m  and  the 
standard  deviation  was  0.51  m. 
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Figure  15.  Histogram  of  sensor  altitude  above  the  water. 


Prior  to  commencing  survey  operations,  the  HeliMag  system  was  test  flown  to  compare  the  base 
noise  levels  of  the  system  to  the  NRL  system.  Details  of  this  analysis  are  presented  in  Appendix 
A.  The  analysis  showed  that  the  Hughes  500/Sky  DAS  combination  had  an  even  lower  noise 
floor  than  the  Bell/AMTADS  combination.  In  addition,  the  most  significant  noise  source  in  the 
Bell  helicopter  was  from  the  rotor  at  a  frequency  of  6.5  Hz  while  for  the  Hughes  this  frequency 
increased  to  8  Hz.  This  provides  a  greater  separation  in  frequency  between  the  rotor  noise  and 
that  of  the  anomalies  of  interest  (which  lie  at  lower  frequencies),  thus  simplifying  the  noise 
filtering  process. 


The  cross-track  data  density  is  essentially  static  and  is  a  function  of  the  system  geometry.  With 
the  exception  of  isolated  data  gaps  (addressed  above)  the  ‘worst  case’  spacing  is  our  sensor 
spacing  of  1 .5  m.  The  effective  density  is  much  higher  than  this  due  to  the  significant  overlap 
required  to  ensure  (or  at  least  minimize)  data  gaps  due  to  the  inevitable  cross-track  variation  of 
the  helicopter  flight  path.  However,  because  the  density  is  not  uniform,  we  quote  the  ‘worst  case’ 
as  the  data  density  achieved.  Down-track  data  density  is  much  higher  than  the  cross-track  density 
and  is  a  function  of  survey  speed.  At  our  final  sample  rate  of  100  Hz,  the  survey  speeds  of 
8-13  m/s  (15-25  kts)  resulted  in  down-line  data  spacing  of  0.08  -  0.13  m. 
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Table  10.  Performance  Metrics  Confirmation  Methods  and  Results 


Performance 

Metric 

Confirmation  Method 

Expected 

Performance 

Performance 

Achieved 

Technology 

Usage 

Field  experience  using  technology 
during  demonstration 

Relative  ease  of  use 

Pass 

Geo-reference 

position 

accuracy 

Infer  sensor  position  accuracy  from 
position  estimates  of  calibration 
targets  derived  using  dipole  analysis 
of  repeated  data  collection  over 
calibration  targets 

Horizontal  <  0.25m; 
Vertical  <  0.5m 

Horizontal  0.13  m; 
Vertical  0.16  m 

HeliMag  survey 
area  coverage 

Percentage  of  survey  area  coverage 
surveyed 

95% 

99.9% 

Operating 

parameters 

(altitude,  speed, 

overlap, 

production 

level) 

Field  data  logs  will  be  used  to 
calculate  the  operating  parameters 

Altitude:  1-3  m  AGL; 
Speed:  15-20  m/s  (30- 
40  kts); 

Overlap:  10%; 
Production  Level:  300 
acres/day 

Altitude:  Mean  2.5  m 
AGL,  stdev  0.55  m; 
Speed:  8-13  m/s  (15- 
25  kts); 

Overlap:  37%; 
Production  Level:  400 
acres/day 

System  Noise 

The  system  noise  will  be  calculated 
as  the  standard  deviation  of  a  20  sec 
window  of  processed  high  altitude 
data. 

<1  nT 

0.11  nT 

Data 

density/point 

spacing. 

Calculated  based  upon  system  sample 
rate  and  survey  speed  (along  track) 
and  system  geometry  and  survey  line 
spacing  (cross  track). 

0.5  m  along-track; 

1.5  m  cross  track 

0.08  -  0.13  m  along 
track;  1.5  m  cross 
track  (max) 

MEC  parameter 
estimates 

Comparison  of  analysis  results  of 
repeated  data  collected  over 
calibration  targets. 

Size  <0.02  m; 

Solid  Angle  <  10° 

Size:  0.012  m;  Solid 
Angle:  3.6° 
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5.  Cost  Assessment 


5.1.  Cost  Reporting 

Cost  information  associated  with  the  demonstration  of  all  airborne  technology,  as  well  as 
associated  activities,  were  tracked  and  documented  before,  during,  and  after  the  demonstration  to 
provide  a  basis  for  determination  of  the  operational  costs  associated  with  this  technology.  For 
this  demonstration,  Table  1 1  contains  the  cost  elements  that  were  tracked  and  documented  for 
this  demonstration.  These  costs  include  both  operational  and  capital  costs  associated  with  system 
design  and  construction;  salary  and  travel  costs  for  support  staff;  subcontract  costs  associated 
with  airborne  services,  support  personnel,  and  leased  equipment;  costs  associated  with  the 
processing,  analysis,  comparison,  and  interpretation  of  airborne  results  generated  by  this 
demonstration.  The  cost  of  the  magnetometers  used  for  the  HeliMag  technology  were  used 
through  a  CRADA  with  NRL;  as  such,  the  actual  cost  of  using  the  technology  was  not  captured 
in  this  demonstration.  However,  we  will  estimate  the  true  cost  of  using  this  technology,  in 
addition  to  the  cost  and  performance  of  all  technologies  demonstrated,  in  the  ESTCP  Cost  and 
Performance  Report  submitted  following  this  demonstration.  In  addition,  due  to  the  helicopter 
incident,  the  cost  of  demobilizing  the  helicopter  is  not  included  in  the  cost  reporting.  This  cost 
would  increase  the  cost  per  acre  analysis. 
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Table  11.  Cost  Tracking 


Cost  Category 

Sub  Category 

Details 

Costs  ($) 

Start-Up  Costs 

Pre-Deployment  and 
Planning 

Includes  planning, 
contracting,  site  visit  and 
site  inspection 

$26,917 

Mobilization 

Personnel  mobilization, 
equipment  mobilization, 
and  transportation 

$45,917 

Operating  Costs 

Helicopter  Survey 

Data  acquisition  and 
associated  tasks,  including 
58  hours  of  helicopter 
operation  time 

$135,841 

Demobilization 

Demobilization 

Demobilization,  packing, 
calibration  line  removal 

$23,496 

Data  Processing 
and  Analysis 

Data  Processing 

Initial  and  secondary 
processing  of  data 

$29,414 

Data  Analysis 

Analysis  of  airborne 
magnetometry  datasets 

$13,438 

Management 

Management  and 

Reporting 

Project  related 
management,  reporting 
and  contracting 

$46,489 

Total  Costs 

Total  Technology  Cost 

$321,512 

Acres  Surveyed 

3,388 

Unit  Cost 

$95/acre 

5.2.  Cost  Analysis 

The  major  cost  driver  for  an  airborne  survey  system  is  the  cost  of  aircraft  airtime.  In  terms  of 
tasks,  this  constitutes  a  major  percentage  of  the  data  acquisition  costs-the  single  largest  cost 
item.  In  addition,  mobilization  costs  for  the  helicopter  were  significant.  Generally,  this  is  a 
function  of  distance  from  the  home  base  for  the  aircraft,  equipment  and  personnel.  The 
demobilization  costs  reported  above  do  not  include  the  cost  of  the  helicopter  time  due  to  the 
helicopter  incident;  otherwise,  the  cost  would  be  expected  to  be  $22,412  higher.  Data  processing 
and  analysis  functions  made  up  the  bulk  of  the  remaining  costs. 

Project  management  and  reporting  were  a  significant  cost  for  this  demonstration,  as  the  project 
was  conducted  under  the  WAA-PP  and  required  more  meetings,  travel  and  reporting  than  would 
generally  be  expected  for  a  production  level  survey.  Last,  mobilization  and  demobilization  was  a 
significant  task  in  terms  of  cost.  Generally,  this  is  a  function  of  distance  from  the  home  base  for 
the  aircraft,  equipment  and  personnel. 
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Costs  associated  with  validation  were  not  considered  in  the  cost  analysis,  as  the  validation  was 
conducted  as  part  of  the  WAA-PP. 
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6.  Implementation  Issues 

6.1.  Regulatory  and  End-User  Issues 

The  ESTCP  Program  Office  has  established  a  WAA-PP  Advisory  Group  to  facilitate  interactions 
with  the  regulatory  community  and  potential  end-users  of  this  technology.  Members  of  the 
Advisory  Group  include  representatives  of  the  US  EPA,  State  regulators,  Corps  of  Engineers 
officials,  and  representatives  from  the  services.  ESTCP  staff  have  worked  with  the  Advisory 
Group  to  define  goals  for  the  WAA-PP  and  develop  Project  Quality  Objectives.  As  the  analyzed 
data  from  the  demonstrations  become  available,  the  Advisory  Group  will  assist  in  developing  a 
validation  plan. 

There  will  be  a  number  of  issues  to  be  overcome  to  allow  implementation  of  WAA  beyond  the 
pilot  program.  Most  central  is  the  change  in  mindset  that  will  be  required  if  the  goals  of  WAA 
extend  from  delineating  target  areas  to  collecting  data  that  are  useful  in  making  decisions  about 
areas  where  there  is  not  indication  of  munitions  use.  A  main  challenge  of  the  WAA-PP  is  to 
collect  sufficient  data  and  perform  sufficient  evaluation  that  the  applicability  of  these 
technologies  to  uncontaminated  land  and  their  limitations  are  well  understood  and  documents. 
Similarly,  demonstrating  that  WAA  data  can  be  used  to  provide  information  on  target  areas 
regarding  boundaries,  density  and  types  of  munitions  to  be  used  for  prioritization,  cost  estimation 
and  planning  will  require  that  the  error  and  uncertainties  in  these  parameters  are  well 
documented  in  the  program. 
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Appendix  A:  Comparison  on  NRL  DAS  with  Bell  and  Sky 
Hardware  DAS  with  Hughes  500 

25  August  2006 


Summary 

As  part  of  ESTCP-0535,  Sky  Research  will  deploy  our  HeliMag  system  to  the  Toussaint 
River  demonstration  site  to  collect  approximately  3,300  acres  of  magnetometer  data.  Sky 
Research  intends  to  use  a  Hughes  500  series  helicopter  and  boom  platform  to  collect  data 
using  our  new  hardware  data  acquisition  system  (Sky  DAS).  The  ESTCP  Program  Office 
requested  a  comparison  of  the  noise  characteristics  of  this  new  HeliMag  system  versus 
that  of  the  NRL  AMTADS  system  previously  deployed  on  Bell  206L  series  helicopters 
and  demonstrated  for  the  ESTCP  Wide  Area  Assessment  Pilot  Program.  Because  site 
conditions  and  target  response  amplitudes  are  the  same  for  each  system,  the  relative 
efficacy  of  the  systems  may  be  evaluated  by  a  comparison  of  their  platform/helicopter 
related  noise  characteristics. 

The  two  systems  have  never  been  deployed  over  the  same  area.  Therefore,  for  this 
comparison  we  use  data  collected  during  high  altitude  flights  in  different  areas  and  on 
different  days.  In  both  cases,  we  selected  a  20  second  section  of  data  for  comprehensive 
analysis.  As  the  analysis  reported  here  shows,  the  Hughes  500/Sky  DAS  combination 
has  an  even  lower  noise  floor  than  the  Bell/ AMTADS  combination.  In  addition,  the  most 
significant  noise  source  in  the  Bell  helicopter  was  from  the  rotor  at  a  frequency  of  6.5  Hz 
while  for  the  Hughes  this  frequency  is  increased  to  8  Hz.  This  provides  a  greater 
separation  in  frequency  between  the  rotor  noise  and  that  of  the  anomalies  of  interest 
(which  lie  at  lower  frequencies),  thus  simplifying  the  noise  filtering  process.  We  also 
provide  a  comparison  of  the  heading  effects  imposed  by  the  static  magnetic  signature  of 
each  platform.  These  effects  manifest  themselves  as  maneuver  noise  in  the  raw  magnetic 
data.  For  most  UXO  applications  the  response  period  of  these  effects  is  sufficiently  long 
to  allow  them  to  be  removed  (along  with  gross  geology  and  magnetic  diurnal  variations) 
using  trend  removal  filters.  However  it  is  prudent  to  ensure  that  the  static  signature  of  the 
Hughes  helicopter  is  not  orders  of  magnitude  greater  than  that  of  the  Bell. 

Table  1  summarizes  the  high  altitude  comparisons.  Note  that  sensor  7  for  the  Sky  system 
was  malfunctioning  and  the  noise  floor  of  that  sensor  is  not  reported.  See  the  text  for  a 
discussion  of  the  raw  noise  in  the  Sky  HeliMag  as  it  is  bimodal  and  hence  the  standard 
deviations  listed  below  under  the  “Raw  Sky”  column  may  not  be  reliable. 
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Table  1.  Standard  deviations  of  the  Bell  helicopter  with  NRL  DAS 
_ and  the  Hughes  500  with  the  Sky  DAS _ 


Raw 

Processed 

Raw 

Processed 

NRL 

NRL 

Sky 

Sky 

1 

1.11 

0.13 

0.40 

0.07 

2 

1.48 

0.14 

0.60 

0.09 

3 

1.52 

0.18 

0.80 

0.11 

4 

1.26 

0.22 

0.80 

0.11 

5 

1.36 

0.10 

0.57 

0.09 

6 

0.94 

0.07 

0.34 

0.07 

7 

0.54 

0.06 

NA 

NA 

NRL  DAS  and  Bell  Helicopter 

A  high  altitude  test  flight  of  the  Bell  Long  Ranger  system  with  the  NRL  DAS  was  used 
for  the  analysis  reported  here.  The  raw  and  processed  data  were  supplied  by  Mike  Tuley 
of  IDA.  The  data  were  imported  into  UXOLab  and  areas  with  steep  turns  were  discarded 
(as  the  helicopter  causes  a  strong  heading  dependant  variation  in  the  magnetic  field). 

Line  6  in  Figure  1  was  selected  for  further  analysis  as  this  line  had  sections  of  data  that 
were  free  of  lower  frequency  variations  in  magnetic  field  (possibly  caused  by  intentional 
yaw  maneuvers  from  the  pilot).  The  analysis  reported  below  uses  twenty  seconds  of  data 
between  GPS  times  67195  and  67215  (Figure  2) 


Easting  (m) 

Figure  1:  Plan  view  of  the  high  altitude  data  available  from  the  NRL  AMT  ADS  system 
on  the  Bell  Long  Ranger. 
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Figure  2  :  Raw  NRL  data  from  the  line  selected  for  analysis. 


The  20  seconds  snippets  of  data  for  each  sensor  were  demedian  filtered  to  remove  any 
trends  in  the  data.  The  noise  varies  from  sensor  to  sensor  and  is  generally  greatest  for  the 
middle  sensor  (sensor  4).  A  time-series  of  the  raw  and  processed  data  for  sensor  4  is 
shown  in  Figure  3.  The  raw  data  varies  from  about  -2  to  +2  nT.  The  standard  deviation 
of  the  readings  is  1 .26  nT.  For  the  processed  data  the  variation  is  reduced  by  a  factor  of 
about  5-10,  with  a  unimodal  distribution  and  a  standard  deviation  of  0.22  nT  (Figures  3 
and  4). 

Figure  4  shows  histograms  of  the  raw  and  processed  data  for  sensors  1  to  6.  The 
histograms  of  the  raw  data  are  unimodal  except  for  sensor  4.  As  explained  in  Appendix 
A  the  reason  this  distribution  is  bimodal  is  the  dominant  noise  source  due  to  the  rotor 
noise  at  6.5  Hz  (Figure  5).  This  rotor  noise  is  most  significant  on  sensor  4  which  is  the 
closest  sensor  to  the  motor. 

Note  that  the  rotor  noise  at  6.5  Hz  is  not  completely  suppressed  in  the  processed  data  for 
sensor  4  (Figure  5).  The  power  spectrum  was  obtained  by  breaking  the  2000  samples 
into  8  sections  of  400  samples  long.  Each  of  these  sections  of  data  were  multiplied  with  a 
window  function  (Kaiser-Bessel)  and  the  FFT  was  calculated.  The  power-spectrums  of 
each  of  the  8  sections  were  then  averaged.  This  method  for  spectral  estimation  produces 
smoother  and  more  reliable  spectral  estimates  than  the  standard  periodogram  analysis 
used  in  the  IDA  report.  However,  note  that  the  spectra  estimated  here  and  in  that  report 
correspond  with  the  same  peaks  at  6.5,  ~26,  and  40  Hz  and  exhibit  the  same  overall 
trends  in  the  spectrum. 
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Figure  3:  Detrend  filtered  data  for  sensor  4.  The  top  row  is  minimally  processed,  the 
bottom  row  is  after  standard  AMTADS  noise  filtering. 
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Magnetic  field  (nT) 


Magnetic  field  (nT) 


Sensor  1  Sensor  2  Sensor  3 


Magnetic  field  (nT) 


Magnetic  field  (nT) 


Magnetic  field  (nT) 


Figure  4:  Histograms  of  the  detrend  filtered  NRL  data,  with  minimally  processed  on  top 
and  processed  on  bottom. 
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Figure  5:  Top:  Power  spectrum  of  the  raw  data  for  sensors  1  and  4.  Bottom:  Power 
spectrum  for  sensor  4  of  the  raw  and  processed  AMTADS/Bell  system 
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The  final  aspect  to  consider  is  the  effect  of  the  heading  of  the  helicopter  on  the  magnetic 
field  recorded  by  the  sensors.  As  shown  in  previous  demonstrations,  heading  (and 
pitch/roll)  changes  are  slow  enough  that  they  do  not  interfere  with  the  signals  of  the 
anomalies  of  interest.  A  simple  moving  median  filter  is  able  to  suppress  heading  and 
diurnal  changes  in  the  Earth’s  field.  In  Table  2,  we  compare  the  magnetic  fields  in  at  each 
sensor  before  and  after  4  turns  during  the  high  altitude  flights.  These  should  provide  an 
indication  of  the  effect  of  heading  on  the  field  recorded  by  each  sensor  (although  the 
magnetic  field  also  varies  spatially  between  the  readings  at  different  headings,  so  this 
estimation  process  is  imperfect).  Through  a  180  degree  change  in  direction,  the  field 
changes  by  about  60  nT  (obtained  as  twice  the  average  value  in  Table  2).  In  Figure  6,  we 
plot  the  differences  in  the  magnetic  field  at  the  49  sensor  intersections  that  occur  where 
the  flight  path  crosses.  There  is  up  to  a  40  nT  difference  in  recorded  field  at  this  location 
where  the  heading  varies  by  90  degrees  (this  just  provides  a  second  indication  of  the 
magnitude  of  the  heading  effect). 


Table  2.  Difference  in  the  magnetic  field  at  each  sensor 


at  the  start  and  end  of  each  of  6  turns 


Heading 

Heading 

Sensor 

Sensor 

Sensor 

Sensor 

Sensor 

Sensor 

Sensor 

before 

after 

Heading 

1 

2 

3 

4 

5 

6 

7 

turn 

turn 

change 

(nT) 

(nT) 

(nT) 

(nT) 

(nT) 

(nT) 

(nT) 

134.2 

-149.9 

75.9 

-4.5 

1.4 

9.8 

9.7 

-2.8 

-12.0 

-13.6 

-108.7 

-39.7 

69.1 

58.4 

59.6 

56.5 

45.6 

40.8 

45.5 

49.6 

-13.2 

46.1 

59.3 

-26.2 

-33.7 

-41.1 

-42.7 

-31.8 

-24.1 

-17.7 

56.2 

121.0 

64.7 

-16.8 

-23.3 

-17.2 

4.0 

15.0 

12.5 

8.4 

Avg. 

26.5 

29.5 

31.2 

25.5 

22.6 

23.5 

22.3 

Figure  6:  Differences  in  the  magnetic  field  at  the  49  sensor-sensor  intersections  where 
the  flight-path  crosses  in  Figure  1. 
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Sky  Hardware  DAS  and  Hughes  500  Helicopter 

For  the  Hughes  500/Sky  DAS  system,  we  use  high  altitude  data  collected  over  the 
Ashland  test  plot  on  June  3,  2006  (Figure  7).  We  use  20  seconds  of  data  from  line  6 
between  GPS  times  65624  and  65644  seconds  (Figure  8).  Note  that  sensor  7  was 
malfunctioning  and  data  from  it  is  not  included  here.  The  400  Hz  data  were  low-pass 
filtered  with  a  52  Hz  cut-off  frequency  and  downsampled  to  100  Hz  (equivalent  to  the 
low-pass  filter  and  100  Hz  sample  rate  of  the  AMTADS).  The  data  were  detrend  filtered 
and  for  each  of  the  sensors  the  data  are  mostly  contained  within  a  +/-  1.5  nT  range 
(Figure  9).  Note  that  all  histograms  for  these  minimally  processed  data  are  bimodal,  with 
the  strongest  effect  evident  with  sensor  4  (closest  to  the  rotor-hub  presumably  causing  the 
noise).  As  explained  in  Appendix  A,  the  bimodal  distributions  occurs  because  of  the 
dominant  sinusoidal  noise  source  at  8  Hz  (Figure  1 1).  After  applying  a  7-9  Hz  notch 
filter,  the  system  noise  is  suppressed  considerably  and  the  distribution  is  unimodal 
(Figures  9  and  10).  Table  1  provides  standard  deviations  of  the  minimally  processed  and 
7-9  Hz  notch  filtered  data. 

The  dominant  feature  in  the  power  spectrum  (Figure  1 1)  is  the  peak  at  7-9  Hz.  There  are 
also  other  smaller  peaks  at  16,  20,  36  42  and  48  Hz,  some  of  which  are  harmonics  of  the 
dominant  peak.  Comparing  sensors  1  and  4,  it  is  evident  that  the  rotor  noise  and 
harmonics  are  more  significant  for  sensor  4. 

The  final  aspect  to  consider  is  the  effect  of  the  heading  of  the  helicopter  on  the  magnetic 
field  recorded  by  the  sensors.  To  investigate  the  effects  of  heading  (but  not  pitch/roll)  we 
analyze  each  intersection  of  each  sensor  with  every  other  sensor  in  the  high  altitude 
flights  (Figure  12).  There  are  enough  intersections  with  this  flight  path  that  we  can  fit  a 
function  of  azimuth  to  each  sensor  and  use  that  to  predict  the  effect  of  heading  (Figure 
12).  This  analysis  shows  that  for  a  full  360  rotation  there  will  be  a  ~50  nT  variation  for 
sensor  1  rising  to  a  maximum  variation  of  ~  120  nT  for  sensor  3,  with  all  other  sensors 
having  values  between  50  and  100  nT.  It’s  difficult  to  directly  compare  the  heading 
analysis  for  the  Hughes  and  Bell  helicopters  as  the  flight  plans  were  so  difference  (the 
Hughes  flights  had  many  flight-path  intersections,  the  Bell  only  one,  the  Bell  covered  a 
much  larger  area  and  encountered  spatial  field  variations  on  the  order  of  200  nT).  There 
may  be  a  slightly  smaller  effect  of  heading  in  the  Bell,  but  its  not  significant  (if  the 
Hughes  helicopter  displayed  heading  differences  in  the  100’s  of  nT  there  may  have  been 
a  need  to  take  corrective  action). 
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Figure  7:  Plan  view  of  the  high  altitude  test- flight. 


Figure  8:  The  20  second  data  segment  used  for  the  characterization  of  the  Hughes  530F 
with  the  Sky  DAS. 
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Figure  9:  Detrend  filtered  data  for  sensor  4.  Top  row  is  minimally  processed,  bottom 
row  is  after  applying  a  notch  filter  to  suppress  frequencies  between  7  to  9  Hz. 


Sky  Research,  Inc. 


-A- 1 0- 


8/25/2006 


Helicopter  System  Comparison 


Sensor  1  Sensor  2  Sensor  3 


Sensor  4 

120, - . - . - . - . - 


Magnetic  field  (nT) 


Sensor  5 

140  | - . - . - . - . - 


Magnetic  field  (nT) 


Sensor  6 


Sensor  1 


Sensor  2 


Sensor  3 


Sensor  4  Sensor  5  Sensor  6 


Magnetic  field  (nT)  Magnetic  field  (nT)  Magnetic  field  (nT) 


Figure  10:  Histograms  of  detrend  filtered  data  for  the  Sky  DAS  /  Hughes  combination. 
Top  row  is  minimally  processed,  bottom  row  is  after  applying  a  notch  filter  to  suppress 
frequencies  between  7  to  9  Hz.  See  the  Appendix  for  a  description  of  why  the  minimally 
processed  data  is  bimodal. 
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Figure  11:  Top:  Power  spectrum  of  the  minimally  processed  data  from  sensors  1  and  4. 
Bottom:  Power  spectrum  of  the  minimally  processed  and  notch  filtered  Sky  HeliMag 
data. 
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Magnetometer  heading  error 


Figure  12:  Analysis  of  the  influence  of  the  helicopter  heading  on  magnetic  field.  At  top 
we  show  the  mismatch  in  recorded  field  values  for  each  sensor  at  every  intersection 
point.  At  bottom  we  show  the  heading  correction  for  each  sensor,  along  with  the 
corrected  mismatch  at  each  sensor  point. 
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Appendix  A1 :  Why  are  the  raw  NRL  and  Sky  histograms 
bimodal? 

The  power-spectrum  analysis  revealed  that  the  dominant  frequency  of  the  noise  was 
between  7  to  9  Hz.  This  relates  to  the  rotation  of  the  rotor  in  the  helicopter.  Now  if  we 
subtract  the  notch  filtered  data  (from  7  to  9  Hz)  from  the  minimally  processed  data,  we 
get  the  histogram  in  Figure  A1  below.  This  is  bimodal  with  peaks  at  +/-  1  nT,  whereas 
the  histogram  of  the  notch  filtered  data  was  unimodal  with  zero  mean  (Figure  10  above). 
Thus  we  see  that  the  7-9  Hz  noise  from  the  rotor  is  the  cause  of  the  bimodal  distribution. 
To  see  why  this  is  the  case,  consider  the  histogram  (Figure  A2)  of  a  uniformly  sampled 
sinusoidal  wave  (the  result  for  a  sin  or  cosine  is  identical  as  they  only  differ  by  a  90 
degree  phase  shift).  The  histogram  is  bimodal  with  modes  at  +1  and  -1.  This  occurs 
because  the  maximum  rate  of  change  of  the  sinusoid  occurs  at  zero  amplitude,  with  the 
minimum  rate  of  change  at  the  positive  and  negative  maxima.  Thus,  if  there  is  a  dominant 
sinusoidal  noise  source  the  histogram  of  the  magnetic  field  will  be  bimodal.  The  peaks  of 
the  bimodal  distribution  will  be  at  plus  and  minus  values  equal  to  the  amplitude  of  the 
noise  source. 


Figure  Al:  This  histogram  was  obtained  by  subtracting  the  notch- filtered  data  from  the 
minimally  processed  data  for  the  Sky  HeliMag  system.  Thus,  it  represents  the  effect  of 
the  7-9  Hz  noise. 
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Figure  A2:  Graph  of  a  sinusoidal  signal  and  a  histogram  of  the  sinusoid  when  sampled 
uniformly  (in  this  case  at  sample  interval  of  pi/180). 
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Appendix  A2:  More  detailed  analysis  of  Hughes  500  / 
Sky  DAS  system 


Figure  Bl:  Profiles  of  the  20  second  section  of  Sky  data,  at  a  sample  rate  of  400  Hz. 
Top  row  are  for  raw  and  two  different  notch  filters.  Bottom  row  is  for  low-pass  filter 
with  and  without  7-9  Hz  notch. 


Sky  Research,  Inc. 


A2-1 


4/17/20076 


Helicopter  System  Comparison 


Figure  B2:  Power  spectra  of  the  20  second  section  of  Sky  data,  out  to  the  Nyquist  rate  of 
200  Hz.  Top  row  are  for  raw  and  two  different  notch  filters.  Bottom  row  is  for  low-pass 
filter  with  and  without  7-9  Hz  notch. 
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Figure  B3:  Histograms  of  the  20  second  segment  of  Sky  data  for  different  levels  of 
processing. 
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Table  Bl.  Standard  deviations  of  the  data  for  different  levels  of  processing.  The  last 
row  is  the  closest  to  the  NRL  processing  scheme. 


Processing  level 

Std  dev 

Raw 

5.52 

Notch  60  Hz 

0.96 

Notch  7-9  &  60  Hz 

0.46 

Low  pass  60  Hz 

0.85 

Notch  7-9  and  Low-pass  60 
Hz 

0.14 

Downsample  to  100  Hz 

0.12 
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APPENDIX  B  -  RECOVERED  TARGET  RESULTS 


GPS  ID 

OE_Type 

Orientation 

Attitude 

Narrative 

Sky  ID 

Fit 

Depth 

Fit 

Moment 

Fit 

Size 

fit 

coherence 

TRH- 

101 

155  mm 

Straight 

Down 

Vert 

155  mm  below  2 
Feet  -  Not 
recovered 

101 

2.63 

3.69 

0.191 

0.900 

TRH- 

103 

155  mm 

North 

Hor 

Mud  Bottom 

103 

2.07 

2.46 

0.167 

0.900 

TRH- 

106 

155  mm 

N 

Hor 

Sand  -  Not 
Recovered  - 
Fuzed 

1 

2.25 

18.56 

0.325 

0.975 

TRH- 

107 

155  mm 

N 

Hor 

Sand 

2 

1.88 

17.15 

0.317 

0.983 

TRH- 

109 

155  mm 

N 

Hor 

Sand  /  Mud 

4 

1.81 

11.41 

0.277 

0.972 

TRH- 

110 

155  mm 

Straight 

Down 

Vert 

Mud  -  Not 
Recovered 

5 

1.52 

7.68 

0.243 

0.973 

TRH- 

111 

155  mm 

N 

Hor 

Sand  -  Not 
Recovered  - 
Fuzed 

6 

1.46 

12.51 

0.286 

0.981 

TRH- 

104 

2.75  in 
Rocket 

60  Deg 
Down 

60  Deg 
Down 

Hard  Mud  -  Not 
Recovered  - 
Assumed  Fuzed 

107 

2.00 

6.64 

0.232 

0.956 

TRH- 

105 

2.75  in 
Rocket 

E 

Hor 

Mud  -  Not 
Recovered  - 
Fuzed 

115 

2.51 

3.08 

0.180 

0.935 

TRH- 

100 

NA 

NA 

NA 

Contact  below  2 
feet 

63 

2.44 

19.39 

0.332 

0.971 

TRH- 

102 

NA 

NA 

NA 

Strong  contact 
below  2  feet 

102 

1.28 

1.65 

0.147 

0.855 

TRH- 

114 

NA 

NA 

NA 

No  Contact 

10 

1.13 

16.47 

0.315 

0.988 

TRH- 

115 

NA 

NA 

NA 

No  Contact 

11 

1.13 

7.14 

0.238 

0.981 

TRH- 

117 

NA 

NA 

NA 

No  Contact 

14 

0.92 

20.33 

0.335 

0.993 

TRH- 

108 

Scrap 

E 

Hor 

Sand  -  Metal  Rod 

3 

1.58 

10.56 

0.270 

0.983 

TRH- 

112 

scrap 

W 

Hor 

Sand  - 12  in  dia 
pipe  -  too  big  to 
dig  out 

8 

1.03 

16.86 

0.315 

0.996 

TRH- 

113 

Scrap 

E 

Hor 

Steel  Can  -  Sand 

9 

1.01 

13.30 

0.292 

0.992 

TRH- 

116 

Scrap 

N 

Hor 

Steel  I-beam  -  too 
big  to  dig  out 

13 

0.65 

8.17 

0.248 

0.986 
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